Task-Specific Robot Base Pose Optimization for Robot-Assisted Surgeries

Preoperative planning and intra-operative system setup are crucial steps to successfully integrate robotically assisted surgical systems (RASS) into the operating room. Efficiency in terms of setup planning directly affects the overall procedural costs and increases acceptance of RASS by surgeons and clinical personnel. Due to the kinematic limitations of RASS, selecting an optimal robot base location and surgery access point for the patient is essential to avoid potentially critical complications due to reachability issues. To this end, this work proposes a novel versatile method for RASS setup and planning based on robot capability maps (CMAPs). CMAPs are a common tool to perform workspace analysis in robotics, as they are in general applicable to any robot kinematics. However, CMAPs have not been completely exploited so far for RASS setup and planning. By adapting global CMAPs to surgical procedure-specific tasks and constraints, a novel RASS capability map (RASSCMAP) is generated. Furthermore, RASSCMAPs can be derived to also comply with kinematic access constraints such as access points in laparoscopy. RASSCMAPs are versatile and applicable to any kind of surgical procedure; they can be used on the one hand for aiding in intra-operative experience-based system setup by visualizing online the robot’s capability to perform a task. On the other hand, they can be used to find the optimal setup by applying a multi-objective optimization based on a genetic algorithm preoperatively, which is then transfered to the operating room during system setup. To illustrate these applications, the method is evaluated in two different use cases, namely, pedicle screw placement in vertebral fixation procedures and general laparoscopy. The proposed RASSCMAPs help in increasing the overall clinical value of RASS by reducing system setup time and guaranteeing proper robot reachability to successfully perform the intended surgerie

The Cluttered Environment Picking Benchmark (CEPB) for Advanced Warehouse Automation

Autonomous and reliable robotic grasping is a desirable functionality in robotic manipulation and is still an open problem. Standardized benchmarks are important tools for evaluating and comparing robotic grasping and manipulation systems among different research groups and also for sharing with the community the best practices to learn from errors. An ideal benchmarking protocol should encompass the different aspects underpinning grasp execution, including the mechatronic design of grippers, planning, perception, and control to give information on each aspect and the overall problem. This article gives an overview of the benchmarks, datasets, and competitions that have been proposed and adopted in the last few years and presents a novel benchmark with protocols for different tasks that evaluate both the single components of the system and the system as a whole, introducing an evaluation metric that allows for a fair comparison in highly cluttered scenes taking into account the difficulty of the clutter. A website dedicated to the benchmark containing information on the different tasks, maintaining the leaderboards, and serving as a contact point for the community is also provided

Planning Realistic Force Interactions for Bimanual Grasping and Manipulation

Dexterous robot hands offer a wide variety of grasping and interaction possibilities with objects. In order to select the best grasp, it is critical to count with a reliable grasp quality measure. Traditional grasp analysis methods use quality measures that allow a relative comparison of grasps for the same object, without an associated physical meaning for the resulting quality. The focus of this thesis is to establish an improved grasp analysis method that will result in a quality measure that can be directly interpreted in the force domain. One of the most commonly used grasp qualities is the largest minimum resisted wrench, which indicates the maximum perturbation wrench that a grasp can resist in any direction. Two efficient ways to calculate this quality are identified: (i) incremental grasp wrench space algorithm, and (ii) ray shooting algorithm. However, existing algorithms for such methods make several assumptions to avoid computational complexities in analyzing the 6D wrench space of a grasp. Important properties like hand actuation, realizable contact forces, friction at the contacts, and geometry of the object to be grasped are either neglected or greatly simplified. In this thesis, these assumptions are improved to bring those algorithms closer to reality. In the case of bimanual grasps, the number of contacts significantly increases, which in turn increases the computational complexity of the process. Suitable algorithms to handle a higher number of contacts are also proposed in this thesis. For grasping an object successfully, considering the hand and the object for analysis are necessary but not sufficient requirements. The capabilities of the robotic arm to which the hand is attached are equally important. Different manipulability measures are considered for the arm, corresponding to single and dual hand grasps, and they are later unified with the physically relevant grasp quality to obtain an overall measure of the goodness of a particular grasp. Based on the updated grasp quality, a complete grasp planning architecture is established. It also includes methods for bimanual grasp synthesis and grasp filtering based on properties like collision with the environment and arm reachability. The thesis includes application examples that illustrate the applicability of the approach. Finally, the developed algorithms can be generalized to a different type of force interaction task, namely a humanoid robot balancing with multiple contacts with the environment. A customized ray shooting algorithm is used to find the stability region of a humanoid legged robot standing on an uneven terrain or making multiple contacts with its hands and legs. In contrast to the regular zero-moment point based method, the stability region is found by analyzing the wrench space of the robot, which makes the method independent of the number of contacts or the contact configuration. Different examples show the direct and intuitive interpretation of the results obtained with the proposed method

Planning Realistic Interactions for Bimanual Grasping and Manipulation

This work presents a dual arm grasp planning architecture that includes two relevant aspects often neglected: differences in hand actuation, and realistic forces applicable by the end effectors. The introduction of an actuation matrix allows considering differences in contact forces that can be generated between, for instance, a fully actuated and an underactuated hand. The consideration of realistic forces allows the computation of real magnitudes of forces and torques that can be resisted by the grasped object. The manipulability workspace can also be computed based on the capability maps, thus providing all the possible motions that can be imparted on the grasped object while respecting the dual hand grasp constraints. The joint consideration of these factors allow the selection of a good grasp for a desired bimanual manipulation

Environment-Aware Grasp Strategy Planning in Clutter for a Variable Stiffness Hand

This paper deals with the problem of planning grasp strategies on constrained and cluttered scenarios. The planner sequences the objects for grasping by considering multiple factors: (i) possible environmental constraints that can be exploited to grasp an object, (ii) object neighborhood, (iii) capability of the arm, and (iv) confidence score of the vision algorithm. To successfully exploit the environmental constraints, this work uses the CLASH hand, a compliant hand that can vary its passive stiffness. The hand can be softened such that it can comply with the object shape, or it can be stiffened to pierce between the objects in clutter. A stiffness decision tree is introduced to choose the best stiffness setting for each particular scenario. In highly cluttered scenarios, a finger position planner is used to find a suitable orientation for the hand such that the fingers can slide in the free regions around the object. Thus, the grasp strategy planner predicts not only the sequence in which the objects can be grasped, but also the required stiffness of the end effector, and the appropriate positions for the fingers around the object. Different experiments are carried out in the context of grocery handling to test the performance of the planner in scenarios that require different grasping strategies

On modeling the cardiovascular system and predicting the human heart rate under strain

With the increasing average age of the population in many developed countries, afflictions like cardiovascular diseases have also increased. Exercising has a proven therapeutic effect on the cardiovascular system and can counteract this development. To avoid overstrain, determining an optimal training dose is crucial. In previous research, heart rate has been shown to be a good measure for cardiovascular behavior. Hence, prediction of the heart rate from work load information is an essential part in models used for training control. Most heart-rate-based models are described in the context of specific scenarios, and have been evaluated on unique datasets only. In this paper, we conduct a joint evaluation of existing approaches to model the cardiovascular system under a certain strain, and compare their predictive performance. For this purpose, we investigated some analytical models as well as some machine learning approaches in two scenarios: prediction over a certain time horizon into the future, and estimation of the relation between work load and heart rate over a whole training session

On Modeling the Cardiovascular System and Predicting the Human Heart Rate under Strain

With the increasing average age of the population in many developed countries, afflictions like cardiovascular diseases have also increased. Exercising has a proven therapeutic effect on the cardiovascular system and can counteract this development. To avoid overstrain, determining an optimal training dose is crucial. In previous research, heart rate has been shown to be a good measure for cardiovascular behavior. Hence, prediction of the heart rate from work load information is an essential part in models used for training control. Most heart-rate-based models are described in the context of specific scenarios, and have been evaluated on unique datasets only. In this paper, we conduct a joint evaluation of existing approaches to model the cardiovascular system under a certain strain, and compare their predictive performance. For this purpose, we investigated some analytical models as well as some machine learning approaches in two scenarios: prediction over a certain time horizon into the future, and estimation of the relation between work load and heart rate over a whole training session

A Bin-Picking Benchmark for Systematic Evaluation of Robotic Pick-and-Place Systems

Pick-and-place operations constitute the majority of today’s industrial robotic applications. However, comparability and reproducibility of results has remained an issue that delays further advances in this field. Evaluation of manipulation systems can be carried out at different levels, but for the final application the performance of the overall system is the critical one. This paper proposes a benchmarking framework for pick-and-place tasks, inspired by a typical task in the logistic domain: picking up fruits and vegetables from a container and placing them in an order bin. The framework uses an easy-to-reproduce environment, a publicly available object set, and guidelines for creating scenarios of different complexity. The proposed benchmark is applied to evaluate the performance of four variants of a robotic system with different end-effector

Autonomous Bipedal Humanoid Grasping with Base Repositioning and Whole-Body Control

Autonomous behaviors in humanoid robots are generally implemented by considering the robot as two separate parts, using the lower body for locomotion and balancing, and the upper body for manipulation actions. This paper provides a unified framework for autonomous grasping with bipedal robots using a compliant whole-body controller. The grasping action is based on parametric grasp planning for unknown objects using shape primitives, which allows a generation of multiple grasp poses on the object. A reachability analysis is used to select the final grasp, and also for triggering a base repositioning behavior that locates the robot on a better position for grasping the desired object more confidently, considering all grasps and the uncertainty in reaching the desired position. The whole-body controller accounts for perturbations at any level and ensures a successful execution of the intended task. The approach is implemented in the humanoid robot TORO, and different experiments demonstrate its robustness and flexibility